1. Technical Field
The present invention relates to methods for the automated treatment of data; and more specifically, the invention relates to automated methods for analyzing and utilizing head and eye tracking data to deduce characteristics about the observed subject such as cognitive and visual distraction, as well as quantify work load levels.
2. Background Art
There is significant ongoing research related to driver fatigue, distraction, workload and other driver-state related factors creating potentially dangerous driving situations. This is not surprising considering that approximately ninety-five percent of all traffic incidents are due to driver error, of which, driver inattention is the most common causative factor. Numerous studies have established the relationship between eye movements and higher cognitive processes. These studies generally argue that eye movements reflect, to some degree, the cognitive state of the driver. In several studies, eye movements are used as a direct measure of a driver's cognitive attention level, and alternatively mental workload.
Knowing where a driver is looking is generally accepted as an important input factor for systems designed to avoid vehicular incidents, an in particularly, crashes. By ascertaining where a driver is looking, Human Machine Interaction (HMI) systems can be optimized, and active safety functions, such as forward collision warnings (FWC), can be adapted on the basis of driver-eye orientation and movements. This may be done as an offline analysis of many subjects, or using an online, or real-time algorithm to perhaps adapt such things as FCW thresholds to the current driver state.
As mentioned above, an interesting use for eye movements is in the ergonomics and HMI fields. For instance, such utilization may be made in determining best placements for Road and Traffic Information (RTI) displays, as well as analyzing whether a certain HMI poses less visual demand than another. These types of analysis can, and are made by studying subjects″ eye movements while using the device —HMI. A primary drawback associated with current methods, however, is that there are few, if any, suitable automated tools for performing the analysis; in their absence, resort is commonly made to labor intensive, manual analysis.
A significant problem in current eye movement research is that every research team seems to use their own definitions and software to decode the eye movement signals. This makes research results very difficult to compare between one another. It is desirable to have a standard that defines visual measures and conceptions. ISO 15007 and SAEJ-2396 constitute examples of such standards in that they prescribe in-vehicle visual demand measurement methods and provide quantification rules for such ocular characteristics as glance frequency, glance time, time off road-scene-ahead and glance duration, and the procedures to obtain them. However, the two standards are based on a recorded-video technique, and rely on frame-by-frame human-rater analysis that is both time consuming and significantly unreliable. As the number of various in-vehicle information and driver assistance systems and devices increases, so will the probable interest for driver eye movements and other cognitive indicators. Thus, the need for a standardized, automated and robust analysis method for eye movements exists, and will become even more important in the future.
Certain eye tracking methods and analysis procedures have been statistically verified to the prescriptions of ISO 15007 and SAEJ-2396. These physical portions of the systems can be configured to be neither intrusive nor very environmentally dependent. At least one example is based on two cameras (a stereo head) being positioned in front of the driver. Software is used to compute gaze vectors and other interesting measures on a real-time basis, indicating such things as head position and rotation (orientation), blinking, blink frequency, and degree of eye-openness. Among other important features in this software are the real-time simultaneous computation of head position/rotation (orientation) and gaze rotation; a feature that has never before been available. Also, it is not sensitive to noisy environments such as occur inside a vehicle. Among other things, “noise” in the data has been found to be a significant factor impacting data-quality-degradation due to such things as variable lighting conditions and head/gaze motion.
It may seem that the previous work done in the area of eye tracking related research is reasonably exhaustive. Yet, as progress is made enabling eye tracking to be more robust and portable, this technology area continues to expand. There are, however, not many on-road studies of driving task-related driver characteristics, and to date, there has been no utilization of eye-tracking data on a real-time basis to calculate measures such as visual or cognitive distraction (see FIGS. 16-18). This is at least partially the result of the time consuming nature of manual segmentation and/or technical difficulties related to the non-portability of commonly used eye-tracking systems. However, in studies conducted in laboratory environments, a variety of algorithms have been developed. Many different approaches have been taken using, for example, Neural Networks, adaptive digital filters, Hidden Markov Models, Least Mean Square methods, dispersion or velocity based methods and other higher derivative methods. Many of these methods, however, are based on the typical characteristics of the eye tracker, such as sampling frequency, and do not work well with other such systems.
Heretofore, there has been no standard for defining what driver characteristic(s) are to be measured, and how they are to be measured. There is no standard that refers to the basic ocular segmentations including saccades, fixations, and eye closures. The standard only concerns glances; that is, the incidence of rapid eye movement across the field of vision.
Interestingly, no current methods take into account smooth eye movements or pursuits; that is, purposeful looks away from the driving path such as looking (reading) a road sign as it is passed. In fact, many studies are designed so that smooth pursuits will never occur, such as by assuring that there are no objects to pursue. This avoidance by current research is understandable; it can be difficult to differentiate a smooth pursuit from a saccade or a fixation. These characteristics are rarely mentioned in the literature. Regardless of the reason(s) that these characteristics have not been considered, smooth pursuits are taken into account with regard to the presently disclosed invention(s) because such smooth eye movement does occur quite often under real driving conditions.
Fundamental to driving a vehicle is the necessity to aim the vehicle, to detect its path or heading, and to detect potential collision threats whether they are from objects or events. This road scene awareness is a prerequisite to longitudinal and lateral control of the vehicle. It should be appreciated that road-center is not always straight ahead of the longitudinal axis of the vehicle, but is often off-centerline due to curves that almost always exist in road-ways to greater and lesser degrees. Even so, research shows that drivers tend to look substantially straight ahead (considering reasonable deviations for road-curvature), with their eyes on the road most of the time; that is, about eight-five to ninety-five percent of the time. Still further, prudence tells the average driver that glances away from the road center or travel path are best timed not to interfere with aiming the vehicle, and to coincide with a low probability of an occurrence of unexpected event or object encounter. Even so, the statistics above demonstrate that even prudent drivers are not always attentive to driving demands, nor are they consistently good managers of their own work loads and distractions when driving.
Driving is not a particularly demanding task in most instances. For example, it is estimated that during most interstate driving, less than fifty percent of a driver's perceptual capacity is used. Because of this, drivers often perform secondary tasks such as dialing cellular phones and changing radio channels. When secondary tasks are performed, a timesharing glance behavior is exhibited in which the eyes are shifted back and forth between the road and the task. This temporal sharing of vision is an implication of having a single visual resource. One could say that the road is sampled while performing secondary tasks instead of the opposite. The problem, which induces collisions, is that unexpected things might happen during the interval when the eyes are off the road and reactions to these unexpected events or objects can be seriously slowed.
The new measures and analysis techniques presented herein exploit this fundamental and necessary driving eye-movement behavior of looking straight ahead or on the vehicle path trajectory. The measures give an accurate off-line assessment of the visual impact of performing visually, cognitively, or manually demanding in-vehicle tasks that have been found to be highly correlated with conventional measures. They also enable a comparison with normal driving. The measures presented herein are importantly also suitable for on-line calculation and assessment of this visual impact and thus represent real-time measures that can be used for distraction and work-load detection.